pLMFPPred: a novel approach for accurate prediction of functional peptides integrating embedding from pre-trained protein language model and imbalanced learning

Functional peptides have the potential to treat a variety of diseases. Their good therapeutic efficacy and low toxicity make them ideal therapeutic agents. Artificial intelligence-based computational strategies can help quickly identify new functional peptides from collections of protein sequences and discover their different functions.Using protein language model-based embeddings (ESM-2), we developed a tool called pLMFPPred (Protein Language Model-based Functional Peptide Predictor) for predicting functional peptides and identifying toxic peptides. We also introduced SMOTE-TOMEK data synthesis sampling and Shapley value-based feature selection techniques to relieve data imbalance issues and reduce computational costs. On a validated independent test set, pLMFPPred achieved accuracy, Area under the curve - Receiver Operating Characteristics, and F1-Score values of 0.974, 0.99, and 0.974, respectively. Comparative experiments show that pLMFPPred outperforms current methods for predicting functional peptides.The experimental results suggest that the proposed method (pLMFPPred) can provide better performance in terms of Accuracy, Area under the curve - Receiver Operating Characteristics, and F1-Score than existing methods. pLMFPPred has achieved good performance in predicting functional peptides and represents a new computational method for predicting functional peptides.Comment: 20 pages, 5 figures,under revie

Zero-shot Model Diagnosis

When it comes to deploying deep vision models, the behavior of these systems must be explicable to ensure confidence in their reliability and fairness. A common approach to evaluate deep learning models is to build a labeled test set with attributes of interest and assess how well it performs. However, creating a balanced test set (i.e., one that is uniformly sampled over all the important traits) is often time-consuming, expensive, and prone to mistakes. The question we try to address is: can we evaluate the sensitivity of deep learning models to arbitrary visual attributes without an annotated test set? This paper argues the case that Zero-shot Model Diagnosis (ZOOM) is possible without the need for a test set nor labeling. To avoid the need for test sets, our system relies on a generative model and CLIP. The key idea is enabling the user to select a set of prompts (relevant to the problem) and our system will automatically search for semantic counterfactual images (i.e., synthesized images that flip the prediction in the case of a binary classifier) using the generative model. We evaluate several visual tasks (classification, key-point detection, and segmentation) in multiple visual domains to demonstrate the viability of our methodology. Extensive experiments demonstrate that our method is capable of producing counterfactual images and offering sensitivity analysis for model diagnosis without the need for a test set.Comment: Accepted in CVPR 202

A multicentre single arm phase 2 trial of neoadjuvant pyrotinib and letrozole plus dalpiciclib for triple-positive breast cancer.

peer reviewedCurrent therapies for HER2-positive breast cancer have limited efficacy in patients with triple-positive breast cancer (TPBC). We conduct a multi-center single-arm phase 2 trial to test the efficacy and safety of an oral neoadjuvant therapy with pyrotinib, letrozole and dalpiciclib (a CDK4/6 inhibitor) in patients with treatment-naïve, stage II-III TPBC with a Karnofsky score of ≥70 (NCT04486911). The primary endpoint is the proportion of patients with pathological complete response (pCR) in the breast and axilla. The secondary endpoints include residual cancer burden (RCB)-0 or RCB-I, objective response rate (ORR), breast pCR (bpCR), safety and changes in molecular targets (Ki67) from baseline to surgery. Following 5 cycles of 4-week treatment, the results meet the primary endpoint with a pCR rate of 30.4% (24 of 79; 95% confidence interval (CI), 21.3-41.3). RCB-0/I is 55.7% (95% CI, 44.7-66.1). ORR is 87.4%, (95% CI, 78.1-93.2) and bpCR is 35.4% (95% CI, 25.8-46.5). The mean Ki67 expression reduces from 40.4% at baseline to 17.9% (P < 0.001) at time of surgery. The most frequent grade 3 or 4 adverse events are neutropenia, leukopenia, and diarrhoea. There is no serious adverse event- or treatment-related death. This fully oral, chemotherapy-free, triplet combined therapy has the potential to be an alternative neoadjuvant regimen for patients with TPBC

Memristor-based Spiking Neural Networks

Emerging non-volatile memory devices, known as memristors, have demonstrated remarkable perspective in neuromorphic hardware designs, particularly in spiking neural network (SNNs) hardware implementation. Memristor-based SNNs have been applied in solving tasks (e.g. image classification and pattern recognition) traditionally solved by conventional artificial neural networks (ANNs), and more attempts in varying disciplines are still being made to exploit the potential of this new research topic. To apply memristors in neuromorphic applications (strictly defined as applications using SNNs in this thesis), two pathways can be followed. One starts by characterising and controlling memristor devices by utilising hardware infrastructure, which is later mapped with the application's higher-level functions (e.g. matrix multiplications). Another embeds data-driven memristor models in software simulators to emulate the application with parameters extracted from real devices. This thesis aims to build a cohesive pipeline for bringing memristor-based SNNs to practical use following these two pathways. To achieve this goal, three key designs have been developed. The first one is an FPGA-based digital interface that is part of a memristor characterisation and control system which enables 64-channel parallel read/write operations and high-speed data processing. The control system, developed by the author and two other researchers, not only acts as a testing tool for collecting memristor characteristics but also delivers higher-level functions with memristor arrays in neuromorphic designs. Whilst the thesis focuses on a usage scenario for memristors, this system includes more powerful, versatile testing functionality for other two-terminal emerging memory devices. The FPGA-based interface was developed by the author solely, and it achieves 64-channel level parallelism in the application aspect and complex digital system design and organisation in the engineering aspect. The digital interface is validated by resistor handling and current-voltage sweep experiments. The second design is the first Python-based algorithm-level simulator, NeuroPack, for memristor-based SNNs with a data-driven memristor model. NeuroPack aims to allow users to quickly validate a neuromorphic concept in the pre-hardware design phase. This tool provides a wide range of optional neuron, plasticity, and device models for users to choose from and answers a fundamental question: is the design functional given the knowledge of the memristor switching dynamics, assuming that the rest of the design is functionally perfect? If yes, the design can move ahead towards the next step. Besides, NeuroPack stores internal variables, including membrane voltages, neuron firing history, and memristor states. With a built-in analysis tool, users can analyse and visualise inference results, observe the evolution of weights and membrane voltages and monitor memristor behaviours. A handwritten digit recognition task in the MNIST dataset showcased how NeuroPack assists users in confirming system validation and exploring sensitivity to critical design choices.The third design tries to expand the usage of memristor-based SNNs to high-dimensional large-scale applications. To do so, a bespoke simulation framework extended from the second design is developed. The first sentiment analysis task in the IMDB movie reviews dataset is exhibited with this framework. Two paths are taken to train spiking neural networks with memristor models: 1) by converting a pre-trained artificial neural network (ANN) to a memristor-based SNN, or 2) by directly training a memristor-based SNN. These two paths have two application scenarios: offline classification and online training. By converting a pre-trained ANN to a memristor-based SNN and training the memristor-based SNN directly, we achieve a classification accuracy of 85.88% and 84.86%, respectively, with the equivalent ANN achieving a baseline training accuracy of 86.02%. From ANNs to SNNs and from non-memristive synapses to data-driven memristive synapses, comparable classification accuracy can be achieved in simulation. In addition, investigations of the neural network sensitivity to global parameters such as spike train length, the read noise, and the weight updating stop conditions have also been given. These investigations further suggest that the simulation framework with statistic memristor models that use experimental data for statistical fitting taking the two paths presented in this chapter can help to exploit the potential of incorporating memristor-based SNNs in text classification tasks. In summary, with the aid of the designs presented in this thesis, we envisage the two pathways now are complete to achieve neuromorphic applications with memristors, especially in performing text classification tasks. The thesis is concluded with the achieved contributions and future perspectives toward AI hardware

Dataset supporting University of Southampton Doctoral thesis &#39;Memristor-based Spiking Neural Networks&#39;

Dataset for the thesis &#39;Memristor-based Spiking Neural Networks&#39;. This dataset contains two Github repositories containing the original source code for content in Chapt 4 &amp; 5 of the thesis Related publication: Authors: Huang Jinqi, Stathopoulos Spyros, Serb Alexantrou, Prodromakis Themis, Title: Neuropack. An Algorithm-Level Python-Based Simulator for Memristor-Empowered Neuro-Inspired Computing Journal: Frontiers in Nanotechnology 4. DOI: https://doi.org/10.3389/fnano.2022.851856 </span

The Effect of Health on Labour Supply of Rural Elderly People in China—An Empirical Analysis Using CHARLS Data

In China, due to decades of the &#8216;one-child policy&#8217; and continuous rural-urban labour migration, real population aging in rural areas is increasing more quickly than in urban areas, and the labour inputs in agricultural production are becoming ever more dependent on the elderly. Using CHARLS data, we examine the effect of health on the labour supply of rural elderly people. We construct a latent health stock index (LHSI) to eliminate measurement bias and then use this one-period lagged LHSI and the Heckman two-stage and the Bourguignon-Fournier-Gurand two-stage method to deal with the simultaneous causality of health and labour decisions and sample selectivity in model estimation. The results show that, in the overall level, the labour force participation and work time of rural elderly people increase significantly with the improvement of health. These effects on the males are sharply greater than on the females and are enhanced with age. In the subdivided agricultural and non-agricultural labour supply, health improvement is positively related with labour force participation of rural elderly and brings an employment allocation from agricultural section to non-agricultural section, especially on the males. However, as the work time, these relations are insignificant and invariant with gender and age

Text classification in memristor-based spiking neural networks

Memristors, emerging non-volatile memory devices, have shown promising potential in neuromorphic hardware designs, especially in spiking neural network (SNN) hardware implementation. Memristor-based SNNs have been successfully applied in a wide range of applications, including image classification and pattern recognition. However, implementing memristor-based SNNs in text classification is still under exploration. One of the main reasons is that training memristor-based SNNs for text classification is costly due to the lack of efficient learning rules and memristor non-idealities. To address these issues and accelerate the research of exploring memristor-based SNNs in text classification applications, we develop a simulation framework with a virtual memristor array using an empirical memristor model. We use this framework to demonstrate a sentiment analysis task in the IMDB movie reviews dataset. We take two approaches to obtain trained SNNs with memristor models: (1) by converting a pre-trained artificial neural network (ANN) to a memristor-based SNN, or (2) by training a memristor-based SNN directly. These two approaches can be applied in two scenarios: offline classification and online training. We achieve the classification accuracy of 85.88% by converting a pre-trained ANN to a memristor-based SNN and 84.86% by training the memristor-based SNN directly, given that the baseline training accuracy of the equivalent ANN is 86.02%. We conclude that it is possible to achieve similar classification accuracy in simulation from ANNs to SNNs and from non-memristive synapses to data-driven memristive synapses. We also investigate how global parameters such as spike train length, the read noise, and the weight updating stop conditions affect the neural networks in both approaches. This investigation further indicates that the simulation using statistic memristor models in the two approaches presented by this paper can assist the exploration of memristor-based SNNs in natural language processing tasks

NeuroPack: an Algorithm-Level Python-Based Simulator for Memristor-Empowered Neuro-Inspired Computing

Emerging two-terminal nanoscale memory devices, known as memristors, have demonstrated great potential for implementing energy-efficient neuro-inspired computing architectures over the past decade. As a result, a wide range of technologies have been developed that, in turn, are described via distinct empirical models. This diversity of technologies requires the establishment of versatile tools that can enable designers to translate memristors’ attributes in novel neuro-inspired topologies. In this study, we present NeuroPack, a modular, algorithm-level Python-based simulation platform that can support studies of memristor neuro-inspired architectures for performing online learning or offline classification. The NeuroPack environment is designed with versatility being central, allowing the user to choose from a variety of neuron models, learning rules, and memristor models. Its hierarchical structure empowers NeuroPack to predict any memristor state changes and the corresponding neural network behavior across a variety of design decisions and user parameter options. The use of NeuroPack is demonstrated herein via an application example of performing handwritten digit classification with the MNIST dataset and an existing empirical model for metal-oxide memristors.</p